The More You Know: Using Knowledge Graphs for Image Classification

One characteristic that sets humans apart from modern learning-based computer vision algorithms is the ability to acquire knowledge about the world and use that knowledge to reason about the visual world. Humans can learn about the characteristics of objects and the relationships that occur between them to learn a large variety of visual concepts, often with few examples. This paper investigates the use of structured prior knowledge in the form of knowledge graphs and shows that using this knowledge improves performance on image classification. We build on recent work on end-to-end learning on graphs, introducing the Graph Search Neural Network as a way of efficiently incorporating large knowledge graphs into a vision classification pipeline. We show in a number of experiments that our method outperforms standard neural network baselines for multi-label classification.Comment: CVPR 201

Context-Aware Embeddings for Automatic Art Analysis

Automatic art analysis aims to classify and retrieve artistic representations from a collection of images by using computer vision and machine learning techniques. In this work, we propose to enhance visual representations from neural networks with contextual artistic information. Whereas visual representations are able to capture information about the content and the style of an artwork, our proposed context-aware embeddings additionally encode relationships between different artistic attributes, such as author, school, or historical period. We design two different approaches for using context in automatic art analysis. In the first one, contextual data is obtained through a multi-task learning model, in which several attributes are trained together to find visual relationships between elements. In the second approach, context is obtained through an art-specific knowledge graph, which encodes relationships between artistic attributes. An exhaustive evaluation of both of our models in several art analysis problems, such as author identification, type classification, or cross-modal retrieval, show that performance is improved by up to 7.3% in art classification and 37.24% in retrieval when context-aware embeddings are used

The COMBS survey I : Chemical Origins of Metal-Poor Stars in the Galactic Bulge

19 pages, 5 tables, accepted to MNRASChemistry and kinematic studies can determine the origins of stellar population across the Milky Way. The metallicity distribution function of the bulge indicates that it comprises multiple populations, the more metal-poor end of which is particularly poorly understood. It is currently unknown if metal-poor bulge stars ([Fe/H] <−1 dex) are part of the stellar halo in the inner most region, or a distinct bulge population or a combination of these. Cosmological simulations also indicate that the metal-poor bulge stars may be the oldest stars in the Galaxy. In this study, we successfully target metal-poor bulge stars selected using SkyMapper photometry. We determine the stellar parameters of 26 stars and their elemental abundances for 22 elements using R∼ 47 000 VLT/UVES spectra and contrast their elemental properties with that of other Galactic stellar populations. We find that the elemental abundances we derive for our metal-poor bulge stars have lower overall scatter than typically found in the halo. This indicates that these stars may be a distinct population confined to the bulge. If these stars are, alternatively, part of the innermost distribution of the halo, this indicates that the halo is more chemically homogeneous at small Galactic radii than at large radii. We also find two stars whose chemistry is consistent with second-generation globular cluster stars. This paper is the first part of the Chemical Origins of Metal-poor Bulge Stars (COMBS) survey that will chemodynamically characterize the metal-poor bulge population.Peer reviewedFinal Published versio